Machine learning has generally been thought of and attempted to be implemented in the context of artificial intelligence. Artificial intelligence depends on algorithmic solutions (e.g., a computer program) to replicate particular human acts and/or behaviors. The study of neural networks is a sub-area of artificial intelligence which attempts to mimic certain human brain behavior by using individual processing elements that are interconnected by adjustable connections.
In human cognition, perception and understanding of phenomena happen over time and space. That is, we learn things that have a common cause based on “temporal adjacency”, meaning that two phenomena occur around the same time. For example, if a child always sees lightening and then thunder, the child will learn that both phenomena have a common cause. As temporal perception is such an important mechanism in cognition, the inability to model temporal events has been a fundamental limitation of most traditional machine learning models. To this end, Hierarchical Temporary Memories (HTMs) have been developed to simulate temporal aspects of perception and learning. An HTM is a hierarchical network of interconnected nodes that individually and collectively (i) learn, over space and time, one or more causes of sensed input data, and (ii) determine, dependent on learned causes, likely causes of novel sensed input data.
One use of the HTMs is to learn temporal sequences of sensed input data that have the same cause. These sequences are used in inference to determine causes of new sequences of sensed input data. There are instances, however, when sensed inputs with the same cause do not always happen in the same sequence. Such instances can lead to a large number of similar sequences with the same cause. As learning sequences and their causes is based on the frequency at which the sequences of inputs are sensed, enumerating the frequencies of a large number of sequences can be computationally expensive. Therefore, the number of sequences needs to be limited using temporal pooling in which the sequences are grouped and stored for processing.
In such temporal pooling, the learning of temporal sequences at the HTMs may become non-optimal or inaccurate for various reasons. One cause for such non-optimal or inaccurate learning is incorrect grouping of temporal sequences. Because a child node identifies sequences and groups the sequences based on the input data provided to the child node, the child node may not be able to detect any problems in its grouping. Another reason for non-optimal or inaccurate learning is asynchronous outputting of signals from nodes at the same level. Nodes at the same level may generate output signal at different time frame due to differences in the hardware/software configurations of the nodes or different processing loads at the nodes. The parent node receiving asynchronous output of signals from nodes may identify incorrect sequences unless corrective measures are taken.
Also, the number of co-occurrences (quantization points) that can be processed at a node is restricted because of the limited computation and storage resources. Therefore, the co-occurrences to be processed at a node must be chosen carefully in a learning phase to minimize the use of computation and storage resources while maximizing the performance of the HTM. The desirability of choosing which co-occurrences to refine, merger, or discard, however, may not be apparent without any information about the grouping of the sequences.